3D digital painting

ABSTRACT

A method of digital continuous and simultaneous three-dimensional painting and three-dimensional drawing with steps of providing a digital electronic canvas having at least one display and capable of presenting two pictures for a right eye and a left eye; providing means for creating a continuous 3D virtual canvas by digitally changing a value and sign of horizontal disparity between two images for the right eye and the left eye and their scaling on the digital electronic canvas corresponding to instant virtual distance between the painter and an instant image within the virtual 3D canvas; providing at least one multi-axis input control device allowing digital painting or drawing on the digital electronic canvas; painting within virtual 3D canvas by providing simultaneous appearance of a similar stroke on the images for the right eye and the left eye on the digital electronic canvas.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation-in-part of U.S. application Ser. No.15/229,269 filed Aug. 5, 2016, for 3D DIGITAL PAINTING, which is acontinuation-in-part of U.S. application Ser. No. 14/306,090, filed Jun.16, 2014, for 3D DIGITAL PAINTING, now U.S. Pat. No. 9,440,484, issuedSep. 13, 2016, which is a continuation-in-part of U.S. application Ser.No. 13/116,015, filed May 26, 2011, for 3D DIGITAL PAINTING, now U.S.Pat. No. 8,817,017, issued Aug. 26, 2014, which claims the benefit ofU.S. Provisional Application No. 61/396,649, filed Jun. 1, 2010, for 3DDIGITAL PAINTING, which are incorporated in their entirety herein byreference.

FIELD OF THE INVENTION

This invention relates to digital painting and drawing ofthree-dimensional images, that can be viewed and presented withdifferent types of stereo vision. The present invention in variousembodiments creates a new art of three-dimensional painting, provides apowerful tool in education from childhood education to universitygraduate education and further, provides a tool for science andengineering, in design of mechanical models and in chemical andbiological research and the like. In general, the technology of thepresent invention, in various embodiments, can be used for consumer,educational, professional, environmental, military and otherapplications.

BACKGROUND

There is a need for improved mechanisms for digital drawing, paintingand writing technologies. The information contained in a graphical orpainted image (color or monochromatic) cannot be explained, interpretedor communicated by other means, for example by voice, for inputtingdigital information into a computer nearly as effectively as with thegraphical or painted image.

However, until now all painting art is two-dimensional due to thetwo-dimensional nature of all known canvases (more precisely, drawingsurface, even if it is not a plane but, for example, a cylindrical,spherical or other curved surface). One of the challenges of paintingalways was how to create an image of the third dimension, the illusionof depth of a picture. Although a number of great painters achievedfantastic results in this endeavor, nevertheless the interest inthree-dimensional visual images resulted in art forms such as sculptureand architecture. However, painted sculptures didn't receive wideacceptance. Then, realization of three-dimensional or stereo vision,with the advent of photography, when two photo-cameras, spaced at adistance of about the distance between the human eyes, made twopictures. These pictures are viewed through the stereoscope, which wasinvented in 1838.

Progress in stereo photography (both stereo photographs and stereomotion pictures) has been made over the last 170 years. The most recentimprovements in three-dimensional movies and three-dimensional TV aremaking an interest in these technologies even greater, as they becomeavailable in consumer products.

However, there remains a need for improvements in three-dimensionalpainting mostly because nothing changed in dimensionality of thetwo-dimensional canvas for painting.

There are several challenges to solving the problem of three-dimensionalfree-hand painting and drawing. The first challenge is athree-dimensional canvas for a free-hand painting and drawing on orwithin this canvas.

It is hard to imagine, from an existing technology viewpoint, a kind ofmedia suitable for a true three-dimensional (cube) canvas, transparentand allowing for penetration into the cube with a brush, making a strokeof paint, and removing the brush without disturbing the rest of themedia (i.e., drawing or painting). It sounds like science fiction, atleast from the viewpoint of existing technology and known materialsother than painted sculptures, requiring making of a sculpture, and thenpainting of such sculpture.

As known, depth perception, as visual ability to perceive the world inthree dimensions (three-dimensional), arises from a variety of depthcues. From all the depth cues both monocular and binocular stereopsisfound the most practical applications. Stereopsis is the process invisual perception leading to the sensation of depth from the twoslightly different projections of the world onto the retinas of the twoeyes. The differences in the two retinal images are called horizontaldisparity, retinal disparity, or binocular disparity. The differencesarise from the eyes' different positions in the head.

These two images corresponding to different visions of the right andleft eyes are relatively easy to create by stereo-photography orstereo-movie by taking two pictures simultaneously with two camerasseparated horizontally similar to two separated eyes. However, it ishard to imagine how an artist can paint two paintings of the same imageon two different canvases for two eyes such that they later could beviewed with one of the stereoscopic viewing systems.

As long as art of painting exists, almost all artists have been tryingto perfect techniques for a three-dimensional illusion on atwo-dimensional canvas, demonstrating the long-felt need for athree-dimensional painting technique. Therefore, there is a need forcreating a method and system for three-dimensional painting and drawing.

SUMMARY

A method of digital recording of three-dimensional painted andthree-dimensional drawn images is presented. The method comprises thesteps of: providing an electronic canvas (computer monitor, TV screen,projection screen, display of the mobile device, etc.); providing meansfor three-dimensional digital vision (shutter glasses, switching canvasbetween left and right eye, polaroid glasses, vertical cylinder lensesraster screens, etc.); providing at least one at least two-axis inputcontrol devices allowing digital painting or drawing on the canvas;providing additional at least one at least one-axis input control devicefor virtual changing the position of the canvas along the axis betweenthe painter and the canvas; painting or drawing on the electronic canvasfor each of the virtual positions of the canvas in the third dimensionby changing this position with the at least one at least one-axis inputcontrol device and verifying this position with the means forthree-dimensional digital vision; providing a description in digitalformat of images for right and left eyes on each virtual position of thecanvas and of corresponding positions of the canvas; providing completetwo-dimensional images of the painting for the right and the left eyes;using left and right images for presentation of three-dimensionalpainting by available means for three-dimensional vision.

A corresponding system for digital recording of three-dimensionalpainted and three-dimensional drawn images is also presented. The systemcomprises: an electronic canvas (computer monitor, TV screen, projectionscreen, display of the mobile device, etc.); means for three-dimensionaldigital vision (shutter glasses, switching canvas between left and righteye, etc.); at least one at least two-axis input control device allowingdigital painting or drawing on the canvas; at least one at leastone-axis input control device for virtual changing the position of thecanvas along the axis between the painter and the canvas; means forthree-dimensional image presentation; wherein the system providespainting or drawing on the electronic canvas for each of the virtualpositions of the canvas in the third dimension by changing this positionwith the at least one-axis input control device and verifying thisposition with the means for three-dimensional digital vision; the systemalso provides a description in digital format of images for right andleft eyes on each virtual position of the canvas and of correspondingpositions of the canvas; the system also provides completetwo-dimensional images of the painting for the right and the left eyes;the system provides using left and right images for presentation ofthree-dimensional painting by available means for three-dimensionalvision.

In accordance with a further embodiment, the present invention can becharacterized as a method of digital continuous and simultaneousthree-dimensional painting and three-dimensional drawing, including thesteps of providing a digital electronic canvas having a screen andcapable of presenting two pictures for a right eye and a left eye;providing means for three-dimensional digital vision; providing meansfor three-dimensional image presentation comprising a processor;providing means for continuous changing of a virtual distance betweenthe digital electronic canvas and a painter by digitally changing ahorizontal shifting (disparity) between images for the right eye and theleft eye on the digital electronic canvas corresponding to instantvirtual canvas position; wherein a resolution Δ of continuity ofchanging of the virtual distance Z between the digital electronic canvasand the painter is defined by a size p of a pixel on the digitalelectronic canvas in horizontal direction and by a distance d betweenpupils of a painter's eyes according to an expression: Δ≈2p Z/d;providing at least one three-axis input control device allowing digitalpainting or drawing on the digital electronic canvas; painting on thedigital electronic canvas for any instant virtual positions of thedigital electronic canvas providing simultaneous appearance of a similarstroke on the images for the right and the left eye, wherein asimultaneousness of appearance of the similar stroke on the images forthe right and the left eye is limited by a smallest time interval equalto an inverted frequency of refreshment of frames on the digitalelectronic canvas and wherein a motion for making simultaneous andcontinuous strokes or lines in all three dimensions is providedsimultaneously and continuously by free moving at least one part of apainter body.

In accordance with yet another embodiment, the present invention can becharacterized as a system for digital continuous and simultaneousthree-dimensional painting and three-dimensional drawing and digitalrecording of three-dimensional painted and three-dimensional drawnimages including a digital electronic canvas having a screen and capableto presenting two pictures for a right eye and a left eye; means forthree-dimensional digital vision (shutter glasses, polaroid glasses,splitting canvas on two canvases for the right eye and the left eye andswitching the two canvases between the right eye and the left eye, orprojecting both pictures for the right eye and the left eye on thedigital electronic canvas, or projecting pictures for the right eye andthe left eye directly into retinas of corresponding eyes, etc.) whereininstant positions of virtual canvas in three-dimensional virtual spaceare displayed; at least one three-axis input control device allowingdigital painting or drawing on the digital electronic canvas; means forthree-dimensional image presentation; wherein two inputs of at least onethree-axis input control device are used for painting of lines orstrokes along two orthogonal axes X and Y in a plane of the digitalelectronic canvas and a third input is used for painting or drawing thelines or strokes along Z axis between a painter and the digitalelectronic canvas by changing a distance between the painter and avirtual position of the digital electronic canvas along the Z axisallowing making three-dimensional paintings (drawings) equallycontinuous in (XY) plane and (XZ) and (YZ) virtual planes; and wherein amotion captured by the at least one three-axis input control device formaking lines or strokes is provided simultaneously and continuously inall three dimensions by movements of at least one part of a painterbody; and wherein the system also provides a description in digitalformat of images for the right eye and the left eye for every and allinstant virtual positions of the digital electronic canvas and ofcorresponding positions of the digital electronic canvas; and the systemalso provides complete two-dimensional images of a painting for theright eye and the left eye at any stage of painting as accumulation ofall lines and strokes made up to that stage; and the system alsoprovides complete three-dimensional images of the painting bysuperposition of all layers corresponding to all virtual positions ofthe digital electronic canvas and the system provides, usingtwo-dimensional left and right images, ability for presentation ofthree-dimensional painting by available means for three-dimensionalvision.

In accordance with a further embodiment, the present invention can becharacterized as a method of digital continuous and simultaneousthree-dimensional painting, three-dimensional drawing, andthree-dimensional cursor (object, image) navigating, including the stepsof providing a digital electronic canvas having a screen or display, asa physical surface, configured for presenting two images: one for aright eye and the other for a left eye of a painter in front of thecanvas; providing means for three-dimensional digital vision; providingmeans for three-dimensional image presentation comprising a processor;providing means for creating a continuous 3D virtual canvas comprisingthe display's surface of the digital electronic canvas and a volume thatincludes the display's surface by digitally changing a value and a signof horizontal disparity between two images for the right eye and theleft eye and their scaling on the digital electronic canvascorresponding to instant virtual distance between the painter's eyes andan instant image within the virtual 3D canvas; wherein a resolution Δ ofcontinuity of changing of the virtual distance Z between the painter andthe virtual images within 3D virtual canvas is defined by a size p of apixel on the digital electronic canvas in horizontal direction and by adistance d between pupils of the painter's eyes according to anexpression: Δ≈2p Z/d; providing at least one input control devicecomprising: a system of sensors that provide an input information aboutfree 3D motion of at least one part of the painter's body into the atleast one input control device for digital painting or drawing within 3Dvirtual canvas; providing at least one kind of a coupling between atleast part of the at least one input control device and the at least onepart of the painter's body, said coupling chosen from a group consistingof: mechanical coupling, optical coupling, electromagnetic coupling,sound coupling, ultrasound coupling, and a combination of two or morethereof moving the at least one part of the painter's body while thesystem of sensors within the at least one input control device isproviding information for recording change of vectors of mechanicalmotion parameters of the at least one part of the painter's body, saidsystem of sensors provide simultaneous appearance of similar strokes orlines on the images for the right and the left eye for any instantposition within 3D virtual canvas; wherein a simultaneousness ofappearance of said similar strokes or lines on the images for the rightand the left eye is limited by a smallest time interval equal to aninverted frequency of refreshment of frames on the digital electroniccanvas and wherein a motion for making strokes or lines in all threedimensions is provided simultaneously and continuously in all directionsof a 3D virtual canvas by free moving the at least one part of thepainter's body.

In accordance with yet another embodiment, the present invention can becharacterized as a system for digital continuous and simultaneousthree-dimensional painting, three-dimensional drawing, three-dimensionalcursor (object, image) navigating, and digital recording ofthree-dimensional painted and three-dimensional drawn images including,

-   a digital electronic canvas having a screen or display with a    physical surface and configured for presenting two pictures: one for    a right eye and the other for a left eye; means for    three-dimensional digital vision, through which a painter sees 3D    images; at least one multi-axis input control device comprising: a    system of sensors, which provide an input information about free 3D    motion of at least one part of the painter's body into the at least    one multi-axis input control device for digital painting or drawing    3D images: means for at least one kind of a coupling between at    least part of the at least one multi-axis input control device and    at least one part of the painter's body, said coupling chosen from a    group consisting of: mechanical coupling, optical coupling,    electromagnetic coupling, sound coupling, ultrasound coupling, and a    combination of two or more thereof; means for three-dimensional    image presentation comprising a processor; wherein inputs of the at    least one multi-axis input control device are used by the means for    three-dimensional image presentation for painting of lines or    strokes along and around three axes X, Y and Z within XYZ virtual    space; means for creating a continuous 3D virtual canvas comprising    the display's surface of the digital electronic canvas and a volume    that includes the surface by digitally changing a value and sign of    horizontal disparity between two images for the right eye and the    left eye and their scaling on the digital electronic canvas    corresponding to instant virtual distance between the painter's eyes    and an instant image within the virtual 3D canvas; wherein a    resolution Δ of continuity of changing of the virtual distance Z    between the painter and the virtual images within 3D virtual canvas    is defined by a size p of a pixel on the digital electronic canvas    in horizontal direction and by a distance d between pupils of the    painter's eyes according to an expression: Δ≈2p Z/d wherein a motion    captured by the at least one multi-axis input control device is used    by the means for three-dimensional image presentation for making    lines or strokes, and the motion is provided simultaneously and    continuously in all three dimensions by movements of the at least    one part of the painter's body; wherein a simultaneousness of    appearance of said similar strokes or lines on the images for the    right and the left eye is limited by a smallest time interval equal    to an inverted frequency of refreshment of frames on the digital    electronic canvas and wherein a motion for making strokes or lines    in all three dimensions is provided simultaneously and continuously    in all directions within 3D virtual canvas by free moving the at    least one part of the painter's body; wherein the means for    three-dimensional image presentation provides a description in    digital format of images for the right eye and the left eye on the    digital electronic canvas for every and all instant images made    within 3D virtual canvas; wherein the means for three-dimensional    image presentation provides complete two-dimensional images of a    painting for the right eye and the left eye at any stage of painting    as accumulation of all lines and strokes made up to that stage; and    wherein the means for three-dimensional image presentation provides    complete three-dimensional images of a painting including linear and    angular position within 3D virtual canvas as accumulation of all    lines and strokes made to the stage of completion and the means for    three-dimensional image presentation provides two-dimensional left    and right images to the means for three-dimensional digital vision    for presentation of three-dimensional painting.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 shows a core principle of the method of three-dimensional digitalpainting in accordance with one embodiment.

FIG. 2 shows an algorithm of the method for digital recording ofthree-dimensional painted and three-dimensional drawn images inaccordance with a further embodiment.

FIG. 3A shows a concept of three-dimensional digital painting on avirtual canvas located between painter and digital monitor in accordancewith another embodiment and FIG. 3B illustrates even more detailedconcept of three-dimensional painting with multiple input controldevices, different means for stereo image presentation and vision anddifferent kind of coupling between input control devices and differentparts of human body, motion of which result in three-dimensionalpainting.

FIG. 4 illustrates how horizontal disparity between images for the righteye and the left eye on the digital electronic canvas depend oncorresponding instant virtual 2D canvas position relative to thepainter's eyes.

FIG. 5 illustrates how the horizontal disparity between images for theright eye and the left eye on the digital electronic canvas change sign,when virtual 2D canvas changes position from in front of the plane of adisplay of the digital electronic canvas to behind the plane of adisplay.

FIGS. 6A and 6B illustrate the offset (horizontal disparety) of thevirtual canvas for a right and left eye correspondingly, as a functionof the virtual canvas position.

FIG. 7 illustrates how central vertical lines of the canvases for theright and left eyes change (offset) in opposite directions depending onthe virtual position of the canvas.

FIG. 8 illustrates virtual Z-axes and temporary grid indicating theposition of virtual canvas.

FIG. 9 illustrates the concept of a continuous 3D virtual canvas.

FIG. 10 illustrates a schematic top view of an art studio for virtualthree-dimensional painting.

FIG. 11 shows a concept of an end-point digital brush and its majorcomponents.

FIG. 12 shows a three-dimensional force sensor, which can be used inend-point digital brush.

FIG. 13 illustrates the three-dimensional force sensor within anend-point of an interchangeable cartridge of the digital brush.

FIG. 14 shows a concept of a universal tool combining a smart phone anda digital brush and pen, as a universal input control device, aspointing/navigating, hand-written text messaging and drawing messaging.

FIG. 15A is an illustration of a universal tool and FIG. 15 Billustrates a different version of a universal tool, having a touchscreen for navigation, gaming, smart phone, as a universal remotecontrol for the smart-home applications, for making 3D photos, or 3Dvideos.

FIG. 16 illustrates how coordinates of point of focus on XY planeparallel to the canvas can be measured and calculated.

FIG. 17 illustrates how a coordinate Z (a depth in three-dimensionalvirtual space) of point of focus on XZ plane can be calculated.

FIGS. 18A and 18B illustrate a concept of a system and method for taking3D photo and video images with a modified (new) smart phone (FIG. 18A)or universal tool (FIG. 18B) having two photo-cameras, modifying these3D images with 3D photoshop and transmitting corrected pictures.

DETAILED DESCRIPTION

Specific embodiments of the invention will now be described in detailwith reference to the accompanying figures.

In the following detailed description of embodiments of the invention,numerous specific details are set forth in order to provide a morethorough understanding of the invention. However, it will be apparent toone of ordinary skill in the art that the invention may be practicedwithout these specific details. In other instances, well-known featureshave not been described in detail to avoid unnecessarily complicatingthe description.

Objects and Advantages

A feature of the present invention in accordance with some embodimentsis to provide a method of 3-dimensional digital painting, drawing,writing and cursor or object (image) navigating within virtual oraugmented 3-dimensional space.

Another feature of the present invention in accordance with someembodiments is to provide a method of 3-dimensional digital painting,drawing, writing and cursor or object navigating accomplished by motionof at least one part of the painter's body.

Another feature of the present invention in accordance with someembodiments is to provide a method of 3-dimensional digital painting,drawing, writing and cursor or object navigating accomplished bysimultaneous motion of several parts of the painter's body.

Another feature of the present invention in accordance with someembodiments is to provide a method of handless 3-dimensional digitalpainting, drawing, writing and navigating of the cursor or object on themonitor accomplished by motion of at least one part of the painter'sbody other than arms, hands or fingers.

Another feature of the present invention in accordance with someembodiments is to provide the means for three-dimensional imagepresentation comprising a processor.

Another feature of the present invention in accordance with someembodiments is to provide a method of visualizing initially empty3-dimensional virtual or augmented space for digital painting bypresenting of temporary 3D virtual grid created by a system of linesrelated to a chosen system of coordinates within said space, whereinthis grid helps to see where the instant 3D image is located within 3Dcanvas and how it is scaled depending on the virtual distance betweenthe painter's eyes and location of instant image

Another feature of the present invention in accordance with someembodiments is to provide a capability to select, rotate, shift and zooma certain volume of temporary 3D virtual grid created by a system oflines related to a chosen system of coordinates within virtual space ofpainting.

Another feature of the present invention in accordance with someembodiments is to provide a method of 3-dimensional digital painting,drawing, writing and cursor or object or image navigating within virtualor augmented 3-dimensional space in the “walking” mode, when a painteris moving around the created 3D objects within virtual or augmented3-dimensional space.

Another feature of the present invention in accordance with someembodiments is to provide a method of 3-dimensional digital painting,drawing, writing and cursor or image navigating within virtual oraugmented 3-dimensional space in the (“turn-table”) mode, when a painterworks in the comfort of his/her chair while the objects of a project areselected, shifted, rotated, zoomed relative to the painter's field ofview.

Another feature of the present invention in accordance with someembodiments is to provide a method of 3-dimensional digital navigatingof the objects within the virtual or augmented three-dimensional spaceof a computer monitor, mobile gaming device or devices comprisingglasses or headsets for three-dimensional vision.

Another feature of the present invention in accordance with someembodiments is to provide a method of 3-dimensional digital editing ofthe three-dimensional pictures, drawings or photos (three-dimensionalphotoshop).

Another feature of the present invention in accordance with someembodiments is to provide a method of converting 2-dimensional imagesinto 3-dimensional images.

Another feature of the present invention in accordance with someembodiments is to provide a method of digital recording ofthree-dimensional painted and three-dimensional drawn images.

Another feature of the present invention in accordance with someembodiments is to provide a method of 3-dimensional digital painting,drawing, writing and navigating simultaneously by multiple participants,which creates a new platform for virtual three-dimensionalcommunication, education, gaming, design, research and development andentertainment.

Another feature of the present invention in accordance with someembodiments is to provide a system for 3-dimensional digital painting,drawing, writing and cursor or object navigating within virtual oraugmented 3-dimensional space.

Another feature of the present invention in accordance with someembodiments is to provide a system of 3-dimensional digital painting,drawing, writing and cursor navigating accomplished by motion of atleast one part of the painter's body.

Another feature of the present invention in accordance with someembodiments is to provide a system of 3-dimensional digital painting,drawing, writing and cursor navigating accomplished by simultaneousmotion of several parts of the painter's body.

Another feature of the present invention in accordance with someembodiments is to provide a method of handless 3-dimensional digitalpainting, drawing, writing and navigating of the cursor or object on themonitor accomplished by motion of at least one part of the painter'sbody other than arms, hands or fingers.

Another feature of the present invention in accordance with someembodiments is to provide a method of 3-dimensional digital navigatingof the objects within the virtual or augmented three-dimensional spaceof a computer monitor, mobile gaming device or devices, which compriseglasses or headsets for three-dimensional vision.

Another feature of the present invention in accordance with someembodiments is to provide a system for handless 3-dimensional painting,drawing, writing and digital navigating of the features inthree-dimensional computers and mobile devices which is accomplished bymotion of at least one part of the painter's body other than arms, handsor fingers.

Another feature of the present invention in accordance with someembodiments is to provide a system for converting 2-dimensional imagesinto 3-dimensional images.

Another feature of the present invention in accordance with someembodiments is to provide a system for digital recording ofthree-dimensional painted and three-dimensional drawn images.

Another feature of the present invention in accordance with someembodiments is to provide a system for 3-dimensional digital painting,drawing, writing and navigating simultaneously by multiple participants,that creates a new platform for three-dimensional communication,education, gaming, design, research and development and entertainment.

Another feature of the present invention in accordance with someembodiments is to provide input control devices for digitalthree-dimensional painting, three-dimensional drawing, writing andnavigating objects within virtual or augmented 3-dimensional spacecreated by digital electronic canvas and by means for three-dimensionalvision.

Another feature of the present invention in accordance with someembodiments is to provide such input control devices that providerequired number of controlled dimensions, required dynamic range ofstrokes or lines and required accuracy of strokes or lines.

Another feature of the present invention in accordance with someembodiments is to provide voice-controlled input control devices fordigital three-dimensional painting, drawing, writing and navigatingobjects within virtual or augmented 3-dimensional space created bydigital electronic canvas and by means for three-dimensional vision.

Another feature of the present invention in accordance with someembodiments is to provide a system, which is capable to make 3D stereophoto-picture or 3D video, then editing, enhancing, filtering andmodifying three-dimensional photo images (three-dimensional photoshop),and to provide transmitting the edited 3D images via standard channel bythe smart phone.

Another feature of the present invention in accordance with someembodiments is to provide a universal tool, which can be used formultiple applications. It can be used as a painting or drawing tool, asa pen or pencil for writing, as a mouse or joystick for navigation onthe screen of computer or mobile gaming, it can have a wirelesscapability and can be used as a universal remote control of differentelectronically controlled equipment and utilities in the household,smart home or business environment, it can be combined with cell-phonecapabilities with all the attributes, which a smart phone has: displayincluding touch-sensitivity, camera including 3D capabilities,microphone, speakerphone, control buttons, etc.

Another feature of the present invention in accordance with someembodiments is to provide a system, which can be configured fromavailable electronic equipment, as a building blocks, depending on aspecific application, specific tasks, specific characteristics anduser's preferences.

PREFERRED EMBODIMENTS

Principles of the method of digital three-dimensional painting andthree-dimensional drawing illustrated in FIG. 1 and can be described asfollows:

-   providing a digital electronic canvas configured to presenting two    pictures for the right and left eye;-   providing means for creating a continuous 3D virtual canvas by    digitally changing a value and sign a horizontal disparity between    two images for the right eye and the left eye and scaling the images    on the digital electronic canvas corresponding to instant virtual    distance between the painter and an instant image within the virtual    3D canvas;-   digitally painting within 3D virtual canvas for each virtual    position of the canvas providing continuous and simultaneous    appearance of a similar strokes on the right and left images due to    a high spatial and time resolution between right and left image on    the digital electronic canvas.

Based on these principles a method of digital recording ofthree-dimensional painted and three-dimensional drawn images isillustrated in FIG. 2 and is represented by the following sequence ofsteps:

-   providing an electronic canvas (computer monitor, TV screen,    projection screen, display of the mobile device, 3D digital personal    viewer, etc.);-   providing means for three-dimensional digital vision (shutter    glasses, splitting canvas on two canvases and switching these    canvases between left and right eye, 3D digital personal viewer,    etc.);-   providing means for three-dimensional image presentation comprising    a processor;-   providing a continuous 3D virtual canvas showing the current instant    position and size (scale) of the images within the virtual 3D    canvas;-   providing at least one input control device comprising a system of    sensors including 3D motion sensors;-   providing at least one kind of a coupling between at least one input    control device and the at least one part of the painter's body-   painting or drawing within 3D canvas by moving part of painter's    body while recording change of the motion vector components;-   providing a description in digital format of images for right and    left eyes on electronic canvas for each instant moment;-   providing complete two-dimensional images of the painting for the    right and the left eyes;-   using left and right images for presentation of 3D painting by    available means for three-dimensional vision.

This method and corresponding system for realization of this method areillustrated in FIGS. 3A and 3B. The system comprises an electroniccanvas 26 (computer monitor, TV screen, projection screen, display ofthe mobile device, etc.) controlled by computer 14, as shown in FIG. 3A.3D digital personal viewer 5, as shown in FIG. 3B, also serves as anelectronic canvas, which might have its own processor or communicatingwith the computer 14. Means for three-dimensional digital vision requiretwo properties. First, it requires an ability of the screen 26 torepresent two canvases for the right and left eye. It can be done eitherby switching the screen between canvases for the right and the left eye,by projecting two images with different polarization for the right andthe left eye, by making two striped canvases, which overlap in a waywhen columns of pixels on the screen are located in turn belonging totwo different canvases—right and left or to have two micro-displays forthe left and right eye, as in 3D digital personal viewer 5 in FIG. 3B.The second property requires an ability to distinguish correspondingimages for the right and the left eye. It can be realized either byusing shutter glasses 6 (FIG. 3A) synchronized with the switching of thescreen, or by using the raster of vertical miniature cylinder lenses onthe front surface of the screen 26, or using the correspondingpolarizing filters in the positions 8 and 10 on the eye-glasses 6, or tohave two micro-displays for the left and right eye within 3D digitalpersonal viewer 5 (FIG. 3B), or to use autostereoscopic digitaldisplays, which do not require any glasses.

The next key component of the system is at least one input controldevice comprising a system of sensors, which provide an inputinformation about free motion of at least one part of the painter bodyinto input control device for digital painting or drawing on the digitalelectronic canvas. FIG. 3A illustrates two input control devices. One ofthem is electronic brush or pen 16 allowing painting or drawing on thecanvas 26 either by hand and/or fingers 18, as shown in FIG. 3A, or byat least one part of the painter's body other than hand and fingercoupled to the electronic brush or pen. Obviously this brush/pen can bedigital, wireless and provide many other functions useful for inputtinginformation into digital processing systems. For example, in oneembodiment brush 16 comprises at least one three-axis input controldevice allowing digital painting or drawing on the digital electroniccanvas, wherein two inputs of at least one three-axis input controldevice are used for painting of lines or strokes along two orthogonalaxes X and Y in a plane of the digital electronic canvas and a thirdinput is used for painting or drawing the lines or strokes along Z axisbetween a painter and the digital electronic canvas by changing adistance between the painter and an instant virtual position of thedigital electronic canvas along the Z axis allowing makingthree-dimensional paintings (drawings) equally continuous in (XY) planeand (XZ) and (YZ) virtual planes; and wherein a motion captured by theat least one three-axis input control device for making lines or strokesis provided simultaneously and continuously in all three dimensions bymovements of at least one part of a painter body.

In another embodiment brush 16 (FIG. 3A) comprises at least one two-axisinput control device, wherein two inputs of said at least one two-axisinput control device are used for painting of lines or strokes along twoorthogonal axes X and Y in a plane of the digital electronic canvas.Another hand 21 is controlling additional at least one-axis inputcontrol device 20 for changing an instant virtual position of the 2Dcanvas along the axis Z between the painter and the canvas. Simultaneousmotion of both painter's hands provide three-dimensional paintings(drawings) equally continuous in (XY) plane and (XZ) and (YZ) virtualplanes. Although two-axis input control device within the brush 16 issufficient for painting of lines or strokes along two orthogonal axes Xand Y in a plane of the digital electronic canvas increasing a number ofaxes gives many additional features. For example, using three-axis inputcontrol would allow to use it not only like a pen with constant width ofthe line but like a real brush, when a painter will be able to changethe width of the line in the process of painting exactly how he is doingwith the real brush. Moreover, additional controls on the tool mightallow changing the color, transparency, structure of the stroke and manyother characteristics of the painting process and resulting images.

The second input control device 20 in FIG. 3A can have multiple controlbuttons/joysticks 22 can be used for different fingers 24 increasing thenumber of functions to be controlled. The control buttons/joysticks 22can be one-axis, two axis or three-axis additionally increasing thenumber of controlled functions. If thumb and four fingers are used andeach of the buttons/joysticks is a three-axis device then 15 controlfunctions can be used simultaneously. The other parts of the painter'sbody can also be used for this purpose. For example, controllingfunctions of the digital painting process can be chosen from a group of:action, turning on and off, navigation of a cursor on the display of theelectronic device, scrolling, zooming, shadowing, screening, selecting,rotating, deleting, restoring, saving, opening, closing, searching,setting up, previewing, undoing, clearing, repeating, pasting, finding,replacing, inserting, formatting, color selection, color mixing, line orstroke width, brush size, swatch size, sponge size, eraser size, thecanvas virtual position, the depth of the focal plane around the canvasvirtual position, special effects or combination. Naturally, all theseadditional tools and functions can be located within virtual 3D paintingcanvas by dedicating a certain portion 25 of it, as it shown in FIG. 3A.This at least one selected portion of the temporary 3D virtual grid isused as a toolbox for arranging and organizing multiple parameters andcharacteristics of the painting tools, color palette, painting processand its different steps, visual effects, setups, commands, wherein aselectable 3D toolbox is chosen from a group: rolodex, 3D stack ofsliding flat windows, rotating in different directions geometricalshapes like cube, cylinder, prism, pyramid, sphere, ellipsoid, thesurface of which is used for positioning the icons activatingcorresponding actions, 3D tree with a trunk, branches, sub-branches andleaves corresponding to desirable organizational structure. Moreover,each or some of all those virtual tools can be assigned to a specificinput control device coupled to a specific part of a painter's body andparameters of these input control devices such as sensitivity and rangecan be adjusted remotely.

FIG. 3B illustrates more details and variations of the method and systemfor digital continuous and simultaneous three-dimensional painting.Input control device 3 is coupled to at least one part of the painter'sbody 11, wherein a coupling between at least part of the input controldevice and at least one part of the painter body is chosen from thegroup 15: mechanical coupling, optical coupling, electromagneticcoupling, sound coupling, ultrasound coupling or a combination of two ormore thereof. For example, input control devices can be mechanicallycoupled to arm, forearm, hand or fingers, as it is shown by positions 16and 20. Input control device 19 can be coupled to a foot eithermechanically or electro-magnetically respectively comprising, forexample, mechanical sensors or electro-static or electro-magneticproximity sensors. It also can be coupled mechanically to a head andoptically to the eyes of the painter, as it shown in position 5. In thiscase the 3D digital personal viewer 5 combines several functions:function of the digital electronic canvas having a screen and capable topresent two pictures for a right eye and a left eye, function of a 3Dviewer and the function of input control device providing an inputinformation about the motion of the both eyes and about the motion ofthe head, when it comprises at least one multi-axis motion sensor. Ingeneral any part of the human body can be coupled to an input controldevice with one or another kind of coupling or its combination.Generalized input control device 3 wirelessly 17 connected to theprocessor 14 controlling electronic canvas 26 or 5. Obviously multipleinput control devices can be used simultaneously in the complex processof three-dimensional painting and in concert with virtual tool box 25.FIG. 3B also illustrates that the system for digital continuous andsimultaneous three-dimensional painting can be assembled from the piecesof available electronic equipment provided however by support ofcorresponding software. When many parts of the human body are suppliedwith different kind of motion sensors, then such system can be used notonly for the painting purpose but also for biomedical applications inresearch, analysis, medical evaluations and diagnostics, rehabilitation,training in sports, military and competitions because it is capable toprovide recording of two-dimensional and three-dimensional images of themotion of the entire body with all its parts synchronized in time andmutual position in three dimensional space.

In accordance with some embodiments, the parts of the human body mostsuitable for free three-dimensional movement are arms, forearms, handsand fingers thanks to the evolutional genius of biomechanics of bones,joints and muscles of these parts. For many practical applicationsincluding painting two properties of moving parts of the body are mostimportant: the range of the motion in all three dimensions and theabsolute accuracy of the motion. In average the range of motion islargest for an arm and a forearm. Hand has much smaller range of motionand fingers even smaller. The accuracy of the motion is lowest for thearm and the forearm and significantly higher for the hand and especiallyhigh for the fingers. Knowing these properties for centuries paintersartists, when they painted pictures especially on the large canvases,which required the use of a full range of arm and forearm motion forlarge strokes and reaching all the area of the canvas, they also used amaulstick for providing required support for the forearm or hand toincrease accuracy of painting by switching the motion from arm andforearm to motion of hand or fingers. Another challenge of accuratepainting is that small accurate strokes require slow accurate motion. Itrequires very high sensitivity of motion sensors, which is not alwaysachievable. One of the embodiments describes the use of a “digitalmaulstick”. From method viewpoint it requires several steps: freeze theposition of the brush within interested area of three-dimensionalvirtual space, when the brush is moved, for example by free arm; switchthe accuracy of the brush to a higher value by either switching thebrush or adjusting the input control on the brush, which providesrequired accuracy; placing the hand with the brush on supportingsurface, which provides more accurate motion of the hand and thefingers; turn on (unfreeze) the brush motion and continue to paintwithin interested area of three-dimensional virtual space with higheraccuracy. All these steps realize function of digital maulstick.

This system provides painting or drawing on the electronic canvas foreach of the instant virtual positions of the canvas in the thirddimension by changing this position with the at least one-axis inputcontrol device and verifying this position with the means forthree-dimensional digital vision. The system also provides a descriptionin digital format of images for the right and the left eyes on eachvirtual position of the canvas and of corresponding positions of thecanvas The system also provides complete two-dimensional images of thepainting for the right and the left eyes by superposition of all layerscorresponding to virtual positions of the canvas for the right and theleft eye separately The system also provides complete three-dimensionalimages of the painting by superposition of all layers corresponding toall virtual positions of the canvas. The system provides using left andright images for presentation of three-dimensional painting by availablemeans for three-dimensional vision.

As soon as the system will provide complete two-dimensional images ofthe painting for the right and the left eye, any available system forstereoscopic imaging can be used for presentation of three-dimensionalpaintings.

The method of three-dimensional painting is further illustrated in FIG.3A. A painter is painting with an electronic brush 16, which he ismoving, for example, by his hand 18 either on any surface suitable formoving an electronic brush along this surface or in the air. Let ussuppose that a painter is painting a vertical line 30 and he wants topaint it in the virtual position of the canvas 28 in location 31corresponding to a central line between the eyes 2 and 4. This virtualplane 28 is located between the screen 26 and the painter. It means thatboth eyes should be focused on the point 31 to see that line in thedesired position. It means, in turn, that the image of this line for theright eye should be located on the screen 26 in the position 33 and theimage of this line for the left eye should be located on the screen 26in the position 35. Then the painter will be seeing the line 30outstanding from the screen plane 26 to the virtual canvas plane 28 inthe position 31. For achieving this effect the control device 22 shouldprovide the shift of the line 30 image for the right eye on a distance36 into location 32 (33). For the left eye similar shift should be 38 inopposite direction into position 34 (35). In other words, the instantvirtual position of the canvas is determined by the corresponding shiftsof the right and left images in the opposite horizontal directions withrespect to the central line on the screen, as illustrated in FIG. 4. Itcreates binocular disparity and visual sensation of depth.

Changing the shifts of the images for the right eye from 32 to 42 andfor the left eye from 34 to 44 the virtual canvas position will changefrom plane 28 to plane 40, as it is clear from FIG. 4.

FIG. 5 illustrates how the virtual position of the canvas could belocated either in front of the screen or behind the plane of the screen.If the line 42 (43) on the screen 26 is allocated to the virtual screenfor the right eye 4 and the line 44 (45) on the screen 26 is allocatedto the virtual screen for the left eye 2, then the location of thevirtual screen corresponds to plane 40. The image of the line 30 (50)will be located in the position 41 on the virtual plane 40, which islocated between the screen 26 and the painter.

Let us now change the allocations of the images 43 and 45 on the screen26 between the right and left eyes. If the line 42 (43) on the screen 26is allocated to the virtual screen for the left eye 2 and the line 44(45) on the screen 26 is allocated to the virtual screen for the righteye 4, then the location of the virtual screen corresponds to plane 52.The image of the line 30 (50) will be located in the position 53 on thevirtual plane 52, which is positioned behind the screen 26.

It is clear that when the image of the line 30 supposed to be located onthe screen plane in the position 50, then there is no difference betweenthe images for the right and the left eye, i.e. there is no shiftbetween the images for the right and the left eye with respect to thecenter line 31 between the right and the left eye. In this case there isno horizontal or binocular disparity.

FIG. 6A illustrates the function 70 between the desired location of thevirtual 2D canvas plane Z1 or Z2 and the image 64 or 68 correspondinglyon the screen 26 for the right eye 4. Similarly FIG. 6B illustrates thesame for the left eye. This function can be expressed as:Z=Ld/2x,  (1)

-   where Z—location of the virtual canvas;-   L—distance between painter and monitor;    -   d—distance between eyes;    -   x—location of the image on the screen.

Therefore, in order to move the virtual canvas into a coordinate Z onthe axis Z, which originates in the painter eye and perpendicular to thescreen, the control of this function should provide a horizontal shift xof the virtual canvas for this eye relative to the point on the screenwhere axis Z crosses the screen. This shift should be directed towardthe other eye. As a result, two virtual canvases will exist in thepainter's field of vision and due to their mutual shift in oppositedirections the painter will see one virtual screen located either infront or behind the plane of the screen in the coordinate Z. Thisrequired shift x can be determined from (1) as:x=Ld/2Z,

The minimal shift x is determined by a size of the pixel p on thescreen, which defines the resolution of the motion in the plane of thescreen. Therefore, this resolution should also define a resolution Δ ofcontinuity of changing of the virtual distance Z between the digitalelectronic canvas and the painter according to the expression:Δ≈2pZ/d

Symbol≈ was used to reflect that the accuracy of this expression islimited and is in the range of 0.3-0.5%.

FIG. 7 combines FIGS. 6A and 6B presenting both functions 72 and 74 forright and left eye correspondingly.

The challenge with determining the position of the initial canvas isthat it is blank and the right and left eye cannot see the difference intheir images. Similar situation will exist at the beginning stage ofpainting, when there are not too many images or the strokes of theelectronic brush already exist on the canvas. Even at the later stagesof painting or in case of editing three-dimensional pictures(three-dimensional photoshop) it would be very helpful to have temporarygrid or system of lines, which would give a painter or observer thevisual representation of the current position of the virtual canvas.

In the simplest case the virtual position of the canvas is presented byat least one virtual Z-axis having an origin somewhere in the planeparallel to the screen. It might be located at the location of a painterand directed from the painter plane toward the screen. It has a scaledescribing a distance from the painter to the virtual position of thecanvas and this current virtual position of the canvas is indicated onthe scale.

The origin of the virtual Z-axis can be located somewhere in the horizonline and indicate the position of the virtual canvas on the scale of theaxis. In general one virtual Z-axis is presented on the screen as a linewith the scale connecting some point on the periphery of the screen withsome inner point within the screen. The pitch of the scale might benon-linear reflecting the changing distance of the object from thepainter.

In more general cases virtual Z-axis can be presented by the line chosenfrom: straight line, curved line, periodic line, spiral line, verticalline, horizontal line, descending line, ascending line and combination.

It is more convenient to have several Z-axes, as shown in FIG. 8. Inthis case it is possible to show: the position of the virtual canvas;scaling of the similar objects depending on the position of the virtualcanvas; either virtual infinite point—origin of all Z-axes or the mostdistant virtual position of the canvas, both of which can be selected bythe painter. Four major Z-axes 78, 80, 82 and 84 along with severaladditional Z-axes define a three-dimensional grid, as show in FIG. 8.Corresponding points on the major Z-axes are connected by lines 86, 88,94 creating a visual representation of the plane, which can correspondto a current position of the virtual canvas. The temporary grid visuallydefining the virtual position of the canvas is a system of linesincluding lines connecting points on different virtual Z-axescorresponding to the same position of the virtual canvas.

These rectangles 86, 88 and 94 can have additional inner lines 87, 89,which further provide scaling of two-dimensional images on each virtualposition of the canvas. This two-dimensional scaling can be providedautomatically while position of canvas is changing. The function oftwo-dimensional scaling of the canvas position can be chosen beforehandand it will determine the depth of the painting. The depth of thepainting in Z dimension can be controlled depending on requirements to athree-dimensional painting. The Z-axes are perceived by the painter aslines going in Z direction from the painter toward some point behind thescreen or in opposite direction because they are presented on bothcanvases for right and left eyes. Therefore, the entire grid is seen asa three-dimensional structure, within which the painting is happening.

There are a number of options that exist with choosing the location andposition of the axes, their scales, and, as a result, instant virtualpositions of the canvas. For example, at least one virtual Z-axis canhave an angle relative to the perpendicular to the canvas in the rangeof 0 to 90 degrees. The scale on the at least one virtual Z-axis can bechosen as linear, exponential, logarithmic, sine, or any other function.The scale on different virtual Z-axes can be different reflecting thedifferent angular position of the virtual canvas with respect todifferent virtual Z-axes. In another example, there could be two systemsof grids: one is steady coupled to the display or screen and another ismoving coupled to the painter's eyes, which can provide more flexibilityand accuracy in mutual angular positioning of the 3D objects or imagesand their scaling within 3D virtual space

In general a virtual instant current position of the canvas is presentedwithin a virtual space of temporary 3D virtual grid created by a systemof lines related to a chosen system of coordinates within said space,for example Cartesian, Spherical, Cylindrical, Ellipsoidal, etc., andwherein said temporary 3D virtual grid or selected portion of it can berotated or shifted relative to a painter, wherein this temporary 3Dvirtual grid allows painter navigate placement of the strokes or 3Dsketching within initially empty 3D virtual space and also place thestrokes to the sides and to the back of the painted three-dimensionalobject, when the front view of the object is already defined and whereinthis temporary 3D virtual grid can be turned off and on at any time inthe process of painting.

Angular direction of the X and Y axes on the canvas can change fordifferent virtual positions of the canvas or for different points of theat least one virtual Z-axis. It would reflect the possibility ofcreating a distorted three-dimensional virtual space (convex or concave,sort of “crooked mirrors”), which can predetermine the desireddistortion of the images within this space independently of the sequenceof creating/painting those images.

Some of the embodiments for 3D painting were described based on theconcept of painting canvas, which historically and traditionally was atwo-dimensional surface. The idea of painting in third dimensionperpendicular to the plane of the canvas was presented and interpretedas moving a position of the 2D canvas (when it becomes a virtual 2Dcanvas) closer or further from the painter making the entire paintingspace three-dimensional. While this approach of presentation andinterpretation of 3D painting looks productive in explaining theinterrelations between two 2D images for the right and the left eye andstereoscopic virtual 3D image it does not completely reflect the realityand differentiation of invented 3D painting, which is based on a conceptof continuous 3D virtual canvas. FIG. 9 illustrates this concept for twocases of digital electronic canvas: steady 3D digital monitor 12 and 3Ddigital personal viewer 23 with two micro-displays. In one embodimentsteady 3D digital monitor 12 has a display 26, which represent a real 2Dcanvas that actually is two canvases on one display: one is for theright eye and another is for the left eye. These two 2D canvases areseparated one from another either by different intervals of time, wheneach is presented on the display, or by different characteristics likepolarization or color, when they are presented simultaneously andcontinuously on the display 26. As it was explained earlier, 3Dstereoscopic images 41 or 53 can appear either between the painter'seyes 2, 4 and surface 26 (position 41) of the display 12 or behind thedisplay 26, as in position 53, depending on sign and value of thehorizontal disparity between the 2D images on the display 26 for theright and left eye. Schematically the space between the points A, B, Cand D represents a three-dimensional space where the painter can paint3D picture continuously within this 3D virtual canvas. The virtualdistance between AB and CD represents the “depth of the 3D virtualcanvas”, which certainly can be chosen by the painter prior to paintingor changed later. In another embodiment the space of this 3D virtualcanvas can be defined by the temporary 3D grid 29, which schematicallyshown in the FIG. 9 (third dimension, which would be vertical relativeto the painter is not shown). As shown in FIG. 9, the temporary gridreflects how the size of the objects can be scaled depending on theposition within 3D canvas. It also reflects how already pained orexisted objects can be scaled, when they are navigated or moved inanother position within 3D canvas. The side-planes of the 3D virtualcanvas AD and BC define the field of horizontal view, which isdetermined by the size of the screen and distance between the painterhead and the screen 26. The field of view if it is in the range of humanbinocular vision allows painter to create a virtual three-dimensionalpicture within the sector AB and C and D. The field of horizontal viewrepresents the “width of the 3D virtual canvas”, which is obviouslyvariable. Similarly the field of vertical view (not shown in FIG. 9)would represent the “height of 3D virtual canvas”. The limiting borderAB of the 3D virtual canvas is defined by a maximum angle of bringingeyes together. In reality it can be not a flat surface presented by lineAB but a curved surface depending on physiological specifics ofpainter's eyes. The limiting border CD is located on a distance Zmaxfrom the painter, which is the maximum depth of the 3D virtual canvasand it depends on the painter's ability to distinguish the minimalamount n of pixels in horizontal shift of the image on the display 26according to the expression:Zmax=Ld/2n,  (2)where Zmax—the maximum depth of the 3D virtual canvas;

-   -   L—distance between painter and monitor;    -   d—distance between eyes;    -   n—minimal number of pixels on the display that painter is        capable to distinguish in horizontal direction.

According to another embodiment digital electronic canvas 23 inside 3Ddigital personal viewer has two micro-displays 27 and 28 for the rightand for the left eye correspondingly representing two 2D canvases. As 2Ddisplays are located close to the eyes of the painter two additionallenses 8 and 10 are required for focusing images on the displaysdirectly into corresponding eyes. After adjustment of the lenses 8 and10 so that the surface of the displays 27 and 28 will be in the focus ofthe painter's eyes the 3D space between the displays and the glasses isvery limited for using it as part of the 3D virtual canvas. It meansthat the overwhelming majority of the 3D virtual canvas in this casewill be located behind the plane of the micro-displays 27 and 28.Similarly to the previous case 3D stereoscopic image 53 can appearbehind the display 27 and 28, when the real images 48 and 46 appear onthe 2D displays 27 and 28 correspondingly for the right and the lefteyes. The field of horizontal view is determined by the size of thedisplays and distance between the painter head and the displays and alsoby the relation of it to the range of human binocular vision. Themaximum depth of the 3D virtual canvas Zmax from the painter isdetermined by the same expression (2), where parameter L, as distancebetween the painter and the display, is much smaller than in theprevious case and the painter's ability to distinguish the minimalamount n of pixels in horizontal shift of the image on the displays 27and 28 is also different.

In reality (not virtual or augmented) the artists who deal with 3D realobjects are sculptors working with either clay or plasticine. If thesize of the sculpture is big the artist has to walk around it or evenuse latter or scaffolding to reach all the areas of creating sculpture.If the sculpture is not large he usually use a turn-table, which allowshim to sit on one place, rotate table and working on the sculpture fromall sides creating a real three-dimensional object. Similarly in virtualor augmented reality there are two approaches to paint completethree-dimensional picture. Let us look at the FIG. 10, which illustratethe top view of the virtual art studio. In one embodiment painter 11with glasses 5 on his head 99 sits in front of the digital electroniccanvas 26 having a screen and capable of presenting two pictures for aright eye and a left eye. The field of horizontal view 100 is determinedby the size of the screen and distance between the painter head 99 andthe screen 26. The field of view 100 if it is in the range of humanbinocular vision allows painter to create a virtual three-dimensionalpicture within the sector 99, 101, 102. Using one of the possibleversions of the three-dimensional grid 89 the painter can createthree-dimensional object 9 or at least front part of it. In anotherembodiment, when 3D digital personal viewer 5 is used instead of digitalelectronic canvas 26, the painter can walk around virtual object 9continue painting it from sides and back like sculptor walks aroundsculpture in a process of working (“walking” mode). If the painter wantsto work in a comfort of his chair with the digital electronic canvas 26then he can select 3D area 90, 91, 92, 93, which includes object 9, androtate this area around some vertical axis within this area, for exampleclock-wise 98, as shown in FIG. 10. While rotating on some angle he cancontinue to paint his object from sides and back, like sculptor rotatinghis turn-table, to reach all sides of his virtual three-dimensionalpicture (“turn-table” mode). When the work on this object is finished,the painter can return selected area in the initial position andunselect or paste it back in place. Zooming in and out of the selectedarea also liberates the painter from a need to build the scaffoldingaround large three-dimensional picture. Similarly, there are twoapproaches for virtual 360° panoramic three-dimensional painting. One isbased on using 3D digital personal viewer 5, when the painter is walkinginside virtual closed space and painting (“walking” mode). Anotherapproach is based on rotating the entire 360° grid around vertical axisclose to the nose of the painter. From the comfort of his chair thepainter will be seeing and will be able painting within the completeinside virtual closed space sliding sector by sector on the screen ofhis digital electronic canvas 26 (“turn-table” mode).

The total three-dimensional scaling grid, although it is temporary anddoes not interfere with the painting, might be helpful during the entireprocess of painting and can be turned on and off any time. There aremany different ways of visualization of selected virtual position of thecanvas. For example, the selected position of the virtual canvas can bepresented as a focal plane with the small depth of sharpness. It meansthat the images within this plane will be sharp while the images on theother planes in front and behind the selected plane will be presented asblurred. The depth of sharpness might be set up beforehand or controlledcontinuously in the process of painting by one of the controlbuttons/joysticks 22, as shown in FIG. 3A.

In another example of the visualization of selected position of thevirtual canvas this position can be presented as a semi-transparentplane, which is crossed by the Z-axes. The part of Z-axes between theplane and the painter will be seen sharp while the other part, beingbehind the semi-transparent plane, will be seen blurry. Therefore, theposition of the virtual canvas will be visually defined within thechosen coordinate system.

Another approach of creating some kind of 3D structure within a virtual3D painting space, which initially is empty, for visualization of theinstant position of the virtual canvas is using 3D photo-picture. This3D photo-picture can be either borrowed or specifically made for apainting project. This photo-picture can be inserted as an initial 3Dbackground, which can be turned on and off at any moment of the paintingand either linked to the 3D painting or kept independent and separate.Linking this photo-picture to the afterwards painting illustrates theconcept of 3D photoshop, when one makes 3D photo-pictures and needs tocorrect it with manual means. 3D photoshop has many other potentialapplications. For example, one application could be in development ofnew 3D games, based on collective collaborative painting. Anotherexample is personalization of the existing game characters by adjusting,editing, correcting or even complete redrawing or repainting thecharacters to player's taste within the games similar to Pokemon Go oralike.

The method of digital three-dimensional painting, three-dimensionaldrawing and three-dimensional object navigating within 3D virtual canvasby at least one part of the painter's body coupled to the at least oneinput control device and method of digital recording of painted imagesdescribed above can also possess the following properties:

-   -   providing a description in digital format of images on each        virtual position of the canvas and of corresponding positions of        the canvas and providing complete three-dimensional digital        image of the painting for different applications associated with        three-dimensional image processing (like rotation, moving,        scaling, animation, etc.); at least one part of the painter's        body for making said simultaneous and continuous strokes or        lines in all three-dimensions includes any part of a body, which        the painter can move (arm, forearm, hand, fingers, leg, toes,        head, lips, eyes . . . ); providing a system of sensors        comprising at least one sensor, which is remotely determining        (measuring) the motion of at least one part of the painter's        body in at least one dimension;    -   painting or drawing on the 2D electronic canvas for each of the        instant virtual positions of the 2D virtual canvas in the third        dimension will result in simultaneous painting on two 2D        canvases switching between right and left eye with frequency        high enough that hand-painted images would appear on both        canvases with certain horizontal offsets (disparity) with        respect to a central vertical line for each canvas on a distance        corresponding to chosen virtual position of the 2D canvas;        providing simultaneous appearance of a similar stroke for the        images for the right and the left eye, wherein a        simultaneousness of appearance of said similar stroke for the        images for the right and the left eye is limited by a smallest        time interval equal to an inverted frequency of refreshment of        frames on the digital electronic canvas;    -   providing a complete two-dimensional image of the painting for        the right and the left eyes obtained by superposition of all        layers corresponding to virtual positions of the 2D canvas for        the right and the left eye separately;    -   providing continuous chain of multiple virtual positions of the        2D canvas along the axis between the painter and the canvas;    -   providing a property of the virtual 2D canvas to be transparent        outside the painted area on the current virtual position of the        canvas for all images on the next positions of the canvas        located further from the painter;    -   providing a property of the virtual 2D canvas to mask by the        painted area on the current virtual position of the canvas the        images on the next positions of the canvas located further from        the painter;    -   providing such a property of electronic canvas, wherein the        opacity and transparency of the painted areas are controlled        independently for any layer corresponding to an instant position        of the virtual 2D canvas;    -   providing a property of electronic 2D canvas to be switched in        turn for left and right eye with a certain frequency high enough        that drawn lines and painted strokes are simultaneous and        continuous for both right and left images;    -   providing the means for three-dimensional digital vision, which        comprise the electronic canvas capable switching in turn for        left and right eye with a certain frequency and shutter glasses        switching synchronously with electronic canvas between left and        right eye;    -   providing two or more at least two-axis input control devices        allowing simultaneous digital painting or drawing on the canvas;    -   providing two or more at least two-axis input control devices        allowing simultaneous digital painting or drawing on different        virtual positions of the canvas;    -   switching electronic canvas between two or more at least        two-axis input control devices, their corresponding additional        at least one-axis input control devices and corresponding        shutter glasses, which are switching synchronously with        electronic canvas between these input control devices and for        each input control device between left and right eye;    -   providing additional two or more at least one-axis input control        devices, which are used for controlling functions of the digital        painting process chosen from a group of: action, turning on and        off, navigation of a cursor, image or object on the display of        the electronic device, scrolling, zooming, shadowing, screening,        selecting, rotating, deleting, restoring, saving, opening,        closing, searching, setting up, previewing, undoing, clearing,        repeating, pasting, finding, replacing, inserting, formatting,        color selection, color mixing, line or stroke width, brush size,        swatch size, sponge size, eraser size, special effects or        combination; providing a concept of “digital maulstick”, which        provides required accuracy by placing the hand with the brush on        supporting surface for more accurate motion of the hand and the        fingers within interested area of three-dimensional virtual        space.    -   providing painting or drawing on the electronic canvas, which is        happening simultaneously with changing the virtual positions of        the 2D virtual canvas in the third dimension by changing this        position with the at least one-axis input control device and        verifying this position with the means for three-dimensional        digital vision;    -   providing an additional input control determining the range of        variations of the canvas virtual positions within the system of        three-dimensional digital vision;    -   providing an additional input control determining the depth of        the focal plane around the canvas virtual position within the        system of three-dimensional digital vision;    -   providing presentation of the virtual position of the canvas as        a focal plane with a small focal depth for the purpose of        increasing accuracy of verification of virtual canvas position;    -   providing presentation of the virtual position of the canvas as        a temporary grid corresponding to this virtual position of the        canvas;    -   providing a grid in the plane of the canvas corresponding to the        virtual position of the canvas within the system of        three-dimensional digital vision, which has the pitch of the        grid proportional to the distance from the painter to the        virtual position of the canvas;    -   providing grid in the plane of the canvas, wherein pitch of the        grid is proportional to the distance from the painter to the        virtual position of the canvas and changing the virtual position        of the canvas automatically changes the grid pitch;    -   providing such a way of presentation of the virtual position of        the canvas, where it is presented by at least one virtual Z-axis        having an origin somewhere in the plane parallel to the canvas        and located at the location of a painter, this at least one        virtual Z-axis is directed from the painter plane toward the        canvas and having a scale describing a distance from the painter        to the virtual position of the canvas and this virtual position        of the canvas is indicated on this scale;    -   providing at least one virtual Z-axis, which is presented on the        canvas as a line with the scale connecting some point on the        periphery of the canvas with some inner point within the canvas;    -   providing at least one virtual Z-axis, which is presented by the        line chosen from: straight line, curved line, periodic line,        spiral line, vertical line, horizontal line, descending line,        ascending line and combination;    -   providing such angular direction of the X and Y axes on the        canvas, which can change for different virtual positions of the        canvas or for different points of the at least one virtual        Z-axis;    -   providing at least one virtual Z-axis, which can have an angle        relative to the perpendicular to the canvas in the range of 0 to        90 degrees;    -   providing a scale on the at least one virtual Z-axis, which is        chosen from: linear, exponential, logarithmic, sine, or any        other function;    -   providing two virtual Z-axes having an origin somewhere in the        plane parallel to the canvas and located at the location of a        painter, these two virtual Z-axes are directed from the painter        plane toward the canvas and having scales describing a distance        from the painter to the virtual position of the canvas and this        virtual position of the canvas is indicated on these scales;    -   providing three virtual Z-axes having an origin somewhere in the        plane parallel to the canvas and located at the location of a        painter, these three virtual Z-axes are directed from the        painter plane toward the canvas and having scales describing a        distance from the painter to the virtual position of the canvas        and this virtual position of the canvas is indicated on these        scales;    -   providing four virtual Z-axes having an origin somewhere in the        plane parallel to the canvas and located at the location of a        painter, these four virtual Z-axes are directed from the painter        plane toward the canvas and having scales describing a distance        from the painter to the virtual position of the canvas and this        virtual position of the canvas is indicated on these scales;    -   providing scales on different virtual Z-axes, which can be        different reflecting the different angular position of the        virtual canvas with respect to different virtual Z-axes;    -   providing a temporary grid visually defining the virtual        position of the canvas, which is a system of lines including        lines connecting points on different virtual Z-axes        corresponding to the same position of the virtual canvas;    -   providing a temporary 3D virtual grid created by a system of        lines related to a chosen system of coordinates within said        space, for example Cartesian, Spherical, Cylindrical,        Ellipsoidal, etc.;    -   providing that temporary 3D virtual grid or selected portion of        it can be rotated or shifted relative to a painter, which allows        painter to place the strokes to the sides and to the back of the        painted three-dimensional object;    -   providing another kind of 3D structure within a virtual 3D        painting space for visualization of the instant position of the        virtual canvas by using 3D photo-picture;    -   providing 3D Photoshop for correcting and adjusting 3D        photo-pictures;    -   providing a concept of taking the 3D photo-pictures, while        looking how this 3D picture would look like with the 3D viewer        and later using the same equipment for correcting these pictures        with 3D Photoshop;    -   providing means for presentation of three-dimensional images,        which are chosen from a group of: optical projection of two        images with polarized light onto one screen and using polarized        glasses, optical projection of two images with colored light        onto one screen and using corresponding colored glasses,        stereoscopic glasses and two photographs or slides,        three-dimensional digital personal viewer, shutter glasses and        synchronous switching display between left and right eye, direct        projection of left and right images onto retina of the eyes;    -   associating each virtual position of the canvas with the        corresponding zooming value of the image, which can be set up in        different required ranges in such a way that equal size of an        object on different virtual positions of the canvas would be        zoomed out while virtual positions of the canvas will be moving        from the painter and would be zoomed in while virtual positions        of the canvas will be moving toward the painter;    -   associating a painted stroke or image with a certain virtual        position of the canvas and later moving it to a different        virtual position of the canvas, either with zooming or without        zooming, and pasted on this new position of the canvas;    -   associating a painted three-dimensional image with certain range        of virtual positions of the canvas and later moving it to a        different range of virtual positions of the canvas, either with        corresponding zooming or without zooming, and pasting on these        new positions of the canvas.

The system for digital three-dimensional painting, three-dimensionaldrawing and three-dimensional navigating within 3D virtual canvas by atleast one part of the painter's body coupled to the at least one inputcontrol device and system for digital recording of painted imagesdescribed above can also comprise the following elements:

-   an electronic canvas, which is chosen from the group: computer    monitor, TV screen, projection screen, display of the mobile device    or any other graphical computer output device;-   at least two-axis input control device allowing digital painting or    drawing within 3D virtual canvas, which is chosen from the group:    sensitive pad, mouse, track ball, joystick, finger joystick,    thimble, key-button, touch pad, touch screen, 6D motion sensor,    smart pen, universal tool described below or at least one sensor,    which is remotely determining (measuring) the motion of at least one    part of the painter's body in at least one dimension;-   at least two-axis input control device allowing digital painting or    drawing on the canvas, which is a three-axis input control device;-   a three-axis input control device allowing digital painting or    drawing on the canvas, which is a universal tool described below;-   an additional at least one at least one-axis input control device    for virtual changing the position between the painter and the stroke    or line within 3D virtual canvas, which is chosen from a group of:    scroll wheel, sensitive pad, mouse, track ball, joystick, finger    joystick, thimble, key-button, touch pad, touch screen, 6D motion    sensor, smart pen, universal tool described below or at least one    sensor which is remotely determining (measuring) the motion of at    least one part of the painter's body in at least one dimension, and    alike;-   an additional at least one at least one-axis input control device    for virtual changing the position between the painter and the stroke    or line within 3D virtual canvas, which can be either two-axis or    three-axis input control device;-   an additional at least one at least one-axis input control device    for virtual changing the position of the virtual canvas is    integrated within one device with at least one of at least two-axis    input control devices allowing digital painting or drawing on the    virtual canvas;-   an additional input control determining the range of variations of    the virtual canvas positions;-   an additional input control determining the depth of the focal plane    around the canvas virtual position;-   means for presentation of three-dimensional images, which are chosen    from a group of: optical projection of two images with polarized    light onto one screen and using polarized glasses, optical    projection of two images with colored light onto one screen and    using corresponding colored glasses, stereoscopic glasses and two    photographs or slides, three-dimensional digital personal viewer,    shutter glasses and synchronous switching display between left and    right eye, direct projection of left and right images onto retina of    the eyes;-   three-dimensional digital personal viewer, which comprises: two    micro-displays for left and right eye, two lenses of the glasses for    projecting the images from the displays into eyes, processor,    memory, sensors, power supply, transceiver, body of the viewer    integrating all the elements of the device.

FIG. 11 illustrates an example of an input control device allowingdigital painting or drawing on the canvas. This device comprises severalcomponents. An end-point 244 is coupled to an integrated at least one atleast two-axis force sensor 228 within the device; IC circuit fordigitizing the information from at least one at least two-axis forcesensor and processing the data related to the change of the force vectorcomponents. This circuit can be integrated on the same chip 228 withforce sensor. It also can be located within the block 236 in the mainbody 240 of the device. The same block 236 can comprise hardware andsoftware for providing a description in digital format of how the devicehas been pressed to the surface based at least in part on the forceapplied by the end-point to the writing surface. It also can comprisehardware and software for providing a description in digital format ofhow the device has been moved over the surface based at least in part onthe correlation between the force vector components and correspondingmovement vector components. End point 244 with force sensor 228 can beintegrated within an interchangeable head 230, which is electricallyconnected by contacts 234 with the main processing block 236. The head230 is the most delicate part of the device, as it has a directmechanical contact with the painting surface and therefore, wear andtear of the end point would require periodic change. Making the head 230interchangeable and having low cost and leaving the most expensivecomponent within the main body 240 of the device allows reducing thetotal cost of ownership of this device. Block 236 can also comprisewireless communication circuits. Power supply 238 is also located in thedevice body 240. One or more additional control buttons 242 with theirsensors 246 can be integrated in the device.

The device is moving with a hand and/or fingers such that the end-point244 is contacting a surface 118 suitable for such movement across thesurface in a process of painting, drawing, writing or cursor navigatingwhile recording with the integrated at least one at least two-axis forcesensor, the change of the vector force representing the motion of thedevice and force applied to the sensor by the end-point.

The painting device can further comprise at least one sensor chosen fromthe group of sensors consisting of: one-axis linear accelerometer,two-axis linear accelerometer, three-axis linear accelerometer, one-axisgyro, two-axis gyro, three-axis gyro, one-axis angular accelerometer,two-axis angular accelerometer, three-axis angular accelerometer,one-axis compass, two-axis compass, three-axis compass, altimeter fordetermining the motion parameters, at which the device has been movedbetween recording of two positions within the working three-dimensionalpainting space.

The end-point 244 can be chosen from the group: a pin [tip] coupled tothe at least one at least one-axis force sensor, a ball contacting withthe at least at least two-axis force sensor, a bunch of fibers coupledto the at least one at least two-axis force sensor.

FIG. 12 illustrates an example of a force sensor, which can be coupledto an end point of the painting device. It comprises a semiconductorsubstrate sensor chip 122, which, in its turn, comprises a frame element124, a rigid island element 128 and an elastic element 126 mechanicallycoupling said frame and said rigid island elements 128. A set of two ormore stress-sensitive IC components 130, 132, 134, 136 are integratedinto the elastic element 126. At least one force-transferring element138 couples the end-point 244 to a rigid island element 128 of a sensordie 122 for transferring the applied external vector force from theend-point through the force-transferring element 138 to the rigid islandelement 128 of the sensor die 122, and thereby generating electricaloutputs from application the external force vector via stress-sensitiveIC components 130-136 positioned in the elastic element 126 eachcarrying the unique orthogonal system component of signal from the forcevector, where the IC component outputs are functions of input componentstresses developed from the orthogonal system force components.

In general, at least two-axis force sensor is chosen from a group:mechanical force sensor, linear acceleration force sensor, angularacceleration force sensor, Coriolis force or angular rate force sensorand combination.

FIG. 13 illustrates an example of the interchangeable head 350 of thepainting tool. The head comprises an end-point 342 fixed in the conicalpart 348 of the body of the head 350. The force sensor die 346 isattached to the end-point by the force-transferring element 368. Thecontact pads 370 of the die 346 are electrically connected by the wires372 with the external electrical contacts 374, which provide electricalconnection of the force sensor with the main processing electroniccircuit located within non-interchangeable part of the painting tool.

The method described above can be used for a method of convertingtwo-dimensional images into digital three-dimensional images, saidmethod comprising:

-   providing an electronic canvas (computer monitor, TV screen,    projection screen, etc.);-   providing a digitized two-dimensional image (picture, photo,    drawing, etc.)-   providing means for three-dimensional digital vision (shutter    glasses, splitting canvas on two canvases and switching these    canvases between left and right eye, etc.);-   providing at least one at least two-axis input control device    allowing selecting images and digital painting or drawing on the    canvas;-   providing additional at least one at least one-axis input control    device for virtual changing the position of the canvas along the    axis between the painter and the canvas;-   selecting certain areas on the digital two-dimensional image, which    are supposedly closer to the viewer than the background of the    image;-   providing a description in digital format of selected areas;-   selecting virtual position of the canvas, which supposedly    corresponds to the spatial position of the selected areas in the    third dimension between the viewer and the background;-   pasting the selected areas onto the selected virtual position of the    canvas, which will result in splitting the selected areas on two    copies for the right eye and the left eye and shifting (offsetting)    these two copies horizontally in opposite directions with respect to    a central vertical line for each canvas on a distance corresponding    to the chosen virtual position of the canvas;-   selecting next areas on the image, which are supposedly closer to    the viewer than the previously selected and shifted areas of the    image;-   providing a description in digital format of selected areas;-   selecting next virtual position of the canvas, which supposedly    corresponds to the spatial position of the selected next areas in    the third dimension closer to the viewer;-   pasting the selected next areas onto the selected next virtual    position of the canvas, which will result in shifting these next    areas horizontally in opposite directions with respect to a central    vertical line for each canvas on a distance corresponding to the    chosen next virtual position of the canvas;-   selecting next areas on the image and pasting them on the virtual    canvases closer and closer to the viewer as many times as needed    until closest to the viewer areas would not be selected and pasted;-   providing a description in a digital format of composed right and    left images;-   using left and right images for presentation of corresponding    three-dimensional images by available means for three-dimensional    vision.

The sequence of selecting and pasting areas from the image can berealized in an opposite order, namely from the foreground toward thebackground or in arbitrary order.

The areas between the two overlapping consecutively selected areas aregradually shifted horizontally such that pixels closer to the border ofthe first selected area are shifted to the same distance as the firstarea and that pixels closer to the border of the second selected areaare shifted to the same distance as the second area while all the otherpixels between the first and the second areas along the horizontal axisare shifted on different distances according to the predetermined law ofthe gradient of shift between the first and the second areas.

The gaps, which appeared between the two consecutively selected andshifted on different distance areas, can be filled in manually with atleast one at least two-axis input control device allowing digitalpainting or drawing or selecting and stamping on the electronic canvas.

Obviously, the method of three-dimensional painting can be used forediting, enhancing, filtering and modifying three-dimensional photoimages (three-dimensional photoshop), said method comprising:

-   providing a three-dimensional photo image;-   providing an electronic canvas (computer monitor, TV screen,    projection screen, display of the mobile device, etc.);-   providing means for three-dimensional digital vision (shutter    glasses, splitting canvas on two canvases and switching these    canvases between left and right eye, eye glasses or headset with two    micro-displays for a right and a left eye for a direct projection of    right and left images into a retina of the corresponding eye. etc.);-   providing at least one at least two-axis input control device    allowing digital painting, drawing or selection areas on the canvas;-   providing additional at least one at least one-axis input control    device for virtual changing the position of the canvas along the    axis between the painter and the canvas;-   choosing the virtual position of the canvas corresponding to the    areas on the three-dimensional photo image, which is supposed to be    edited, enhanced, filtered or modified;-   painting, drawing, editing, enhancing, filtering or modifying    selected areas on the electronic canvas for each of the virtual    positions of the canvas in the third dimension by changing this    position with the at least one-axis input control device and    verifying this position with the means for three-dimensional digital    vision;-   providing a description in digital format of images for right and    left eyes on each virtual position of the canvas and of    corresponding positions of the canvas;-   providing complete modified and edited two-dimensional images of the    three-dimensional photo image for the right and the left eyes;-   using left and right images for presentation of edited    three-dimensional photo image by available means for    three-dimensional vision.

FIGS. 14 and 15 illustrate an example of a universal tool, which can beused for multiple applications. It can be used as a painting or drawingtool, as a pen or pencil for writing, as a mouse or joystick fornavigation on the screen or computer or mobile gaming. It can have awireless capability and can be combined with cell-phone capabilitieswith all the attributes, which smart phone has: display, camera,microphone, speakerphone, control buttons, etc.

As follows from FIG. 14, which depicts the front, side and backprojections of the device, it has an end-point 302 coupled to themulti-axis force sensor 308 within an interchangeable cartridge 312. Thetool has an electronic block 248, which comprises digital processor,wireless communication circuits, additional sensors, etc. For example,microphone 250 can be part of the block 248. Inside the body 320 of thedevice a power supply 318, speakerphone 252 and photo-camera 256 canalso be located. A part of the body 320 is a display 260. A number ofcontrol buttons 258, 262, 264, 265 can be situated in convenientlocations on the body 320 of the tool. Different functions, as describedabove, can be assigned to these control buttons. Each of these buttonscan be multi-axis micro-joysticks significantly increasing the totalnumber of functions under control.

FIGS. 15 A and B illustrate examples of a universal tool describedabove. While universal tool shown in FIG. 15 A is functionally an almostexact copy of the device shown in FIG. 14 the tool in FIG. 15 Billustrate a different version of a universal tool, having a differentshape, bigger screen, which can be touch-sensitive with multiple iconsproviding additional control functions not only as painting or drawingtool or as a mouse or joystick for navigation on the screen or computeror mobile gaming, but also as a cell-phone with all the attributes of asmart phone. It might have display, camera, microphone, speakerphone,control buttons, etc. It can be also used as a universal remote controlfor the smart-home applications. This universal tool can have two photocameras instead of one, as shown in FIG. 15 B, for making 3D photos, orselfies, or 3D short videos. These two photo cameras 256 can also belocated on the opposite to the display side and if located at a distancefrom each other equal to an average distance between human eyes thenthis tool can be used making and viewing 3D photos and 3D videos.

It should be understood that the microstructures of the die, structuresof the finger-mice, finger buttons and micro-joysticks and methods oftheir fabrication do not limit the present invention, but onlyillustrate some of the various technical solutions covered by thisinvention. While the invention has been described in detail withreference to preferred embodiments, it is understood that variations andmodifications thereof may be made without departing from the true spiritand scope of the invention.

FIGS. 16 and 17 illustrate an example of how painting or drawing can bedone based on the moving of other than hand and finger parts of thehuman body. In particular, human eyes can be used for this purpose. Oneeye can be used for painting or drawing on the two-dimensional plane,while both eyes provide capability of realizing painting or drawing invirtual three-dimensional space. It can be used for multipleapplications. It can be used for handless painting, drawing or writing,as a mouse or joystick for navigation on the screen of a computer ormobile gaming. It can be used for communication with paralyzed patients,for rehabilitation and education. It can be used as an additionalcommunication channel in parallel with verbal/hearing and manual/tactilechannels.

FIG. 16 illustrates schematically two eyeballs 2 and 4 on the XY plane.An initial center of the pupil of the left eye is located in the originof XY plane and an initial center of the pupil of the right eye islocated on a distance d from the origin on axis X. If a human being islooking at the point of view 400 at the XY plane then his/her eyeballsrotate and the pupils of both eyes would have new position on the XYplane. The left pupil will have new coordinates: X=X_(L) and Y=Y_(L) andthe right pupil will be at: X=X_(R) and Y=Y_(R). These coordinatesdetermine the direction 402 of the left eyeball toward the point offocus 400 and the direction 404 of the right eyeball toward the point offocus 400. It is self-explanatory that if one can measure coordinatesX_(L), Y_(L) and X_(R), Y_(R) then the coordinates X₁, Y₁ of the pointof focus will be also known. Therefore, measuring the instant positionof the eye pupils with respect to their initial position and change oftheir position allows determining the instant point of focus andtrajectory of this point on XY plane, as a result of eye movement.

FIG. 17 illustrates schematically how with the measurements describedabove the depth of vision or the third coordinate on the axis Zperpendicular to the XY plane can also be determined. Two eyeballs 2 and4 are shown here on the XZ plane. The glasses 410 for stereoscopicvision are also shown in the picture. These glasses comprise twophoto-sensor arrays 412 and 414 for the left and right eyescorrespondingly. The images of the eye and the pupil are projected tothe photo-sensor arrays through the optical systems 416 and 418. Thesephoto-sensor arrays allow making measurements of initial pupils'position and their instant positions relative to initial positionsdescribed above. When coordinates X_(L) and X_(R) are measured, thenknowing the geometry of the eyeball, it is easy to determine an angulardirection 406 α_(L) and angular direction 408 α_(R) toward projection ofpoint of focus 400 on XZ plane. It gives the Z₁ coordinate of the pointof focus on XZ plane. As a result of these measurements andcalculations, all three coordinates X₁, Y₁ and Z₁ of instant position ofpoint of focus within three-dimensional virtual space are determined.Moving the instant point of focus by moving both eyeballs allows anyoneto make a three-dimensional trajectory of this movement, e.g., makinghandless painting or drawing possible.

FIGS. 18A and 18B illustrate schematically a concept of systems andmethod for taking 3D photo and video images with either a new version ofsmart phone having two photo-cameras (FIG. 18A) or with universal tool(shown in FIG. 15B) also having two photo-cameras (FIG. 18B). Thesesystems can make 3D photo or video, modifying these 3D images with 3Dphotoshop and sharing corrected pictures via the cell-phone. The systemscomprise an eye glasses or headset 450 for viewing 3D images. It alsocomprises a smart phone 452 or universal tool 453, which has two sides454 and 460 for phone and also 453 and 459 for tool correspondingly. Theside 454 (453) comprises two photo-cameras 456 and 458, which arepositioned along longer side of the phone or tool at the distance fromeach other corresponding to average distance between the human eyes. Thedisplay's side 460 of the phone 452 and side 459 of the universal tool453 can be divided into two micro-displays 462 and 464 for a right and aleft eye correspondingly for a direct projection of right and leftimages 462 and 464 into a retina of the eye. The eye glasses or headset450 have a simple optical lenses 466 and 468 allowing see the 3D stereophoto-picture or 3D video before, after and in the process of shootingthe picture or video. The systems also might have some simpleindividualized adjustments like focusing by changing the distancebetween the lenses 466 and 468 and corresponding images 462 and 464 witha knob or slider 470. Control buttons on the edges of the smart phone orcontrol buttons of the universal tool like end-point 302 coupled to themulti-axis force sensor and micro joystick 258 can be used for multiplefunctions required for 3D photo and video shooting and editing.

A corresponding method for editing, enhancing, filtering and modifyingthree-dimensional photo images (three-dimensional photoshop), canadditionally comprise the following steps: providing a three-dimensionalphoto image; choosing the virtual position of the canvas correspondingto the areas on the three-dimensional photo image, which are supposed tobe edited, enhanced, filtered or modified; painting, drawing, editing,enhancing, filtering or modifying selected areas on the electroniccanvas for each of the virtual positions of the canvas and verifyingthis position with the means for three-dimensional digital vision;providing complete modified and edited two-dimensional images of thethree-dimensional photo image for the right and the left eyes; usingleft and right images for presentation of edited three-dimensional photoimage by available means for three-dimensional vision.

A systems illustrated in FIGS. 18A and B can be used not only for taking3D stereo photo-pictures or 3D video but also for editing, enhancing,filtering and modifying three-dimensional photo images(three-dimensional photoshop) with the same hardware. The obviousadvantage of the system is the ability to share not only limited volumeof downloaded 3D stereo images but unlimited and personal 3D images,which were taken, edited and transmitted via standard channel by thesmart phone used within the system and the recipient can immediatelywatch 3D images using his/her regular phone, which doesn't have twophoto-cameras on the opposite to display's side of the phone, along withlow cost 3D viewer, for example the Google Cardboard Viewer.

Therefore, while the invention has been described with respect to alimited number of the embodiments, those skilled in the art, havingbenefits of this invention, will appreciate that other embodiments canbe devised which do not depart from the scope of the invention asdisclosed herein. Other aspects of the invention will be apparent fromthe following description and the appended claims.

What is claimed is:
 1. A method of digital continuous and simultaneousthree-dimensional painting, three-dimensional drawing, andthree-dimensional cursor (object, image) navigating, said methodcomprising: providing a digital electronic canvas having a screen ordisplay, as a physical surface, configured for presenting two images:one for a right eye and the other for a left eye of a painter in frontof the canvas; providing means for three-dimensional digital vision;providing means for three-dimensional image presentation comprising aprocessor; providing means for creating a continuous 3D virtual canvascomprising the display's surface of the digital electronic canvas and avolume that includes the display's surface by digitally changing a valueand a sign of horizontal disparity between two images for the right eyeand the left eye and their scaling on the digital electronic canvascorresponding to instant virtual distance between the painter's eyes andan instant image within the virtual 3D canvas; wherein a resolution Δ ofcontinuity of changing of the virtual distance Z between the painter andthe virtual images within 3D virtual canvas is defined by a size p of apixel on the digital electronic canvas in horizontal direction and by adistance d between pupils of the painter's eyes according to anexpression: Δ≈2p Z/d; providing at least one input control devicecomprising: a system of sensors that provide an input information aboutfree 3D motion of at least one part of the painter's body into the atleast one input control device for digital painting or drawing within 3Dvirtual canvas; providing at least one kind of a coupling between atleast part of the at least one input control device and the at least onepart of the painter's body, said coupling chosen from a group consistingof: mechanical coupling, optical coupling, electromagnetic coupling,sound coupling, ultrasound coupling, and a combination of two or morethereof; moving the at least one part of the painter's body while thesystem of sensors within the at least one input control device isproviding information for recording change of vectors of mechanicalmotion parameters of the at least one part of the painter's body, saidsystem of sensors provide simultaneous appearance of similar scaledstrokes or lines on the images for the right and the left eye for anyinstant position within 3D virtual canvas; wherein a simultaneousness ofappearance of said similar strokes or lines on the images for the rightand the left eye is limited by a smallest time interval equal to aninverted frequency of refreshment of frames on the digital electroniccanvas and wherein a motion for making strokes or lines in all threedimensions is provided simultaneously and continuously in all directionsof a 3D virtual space by free moving the at least one part of thepainter's body.
 2. A method according to claim 1, wherein free linearand angular motion of the at least one part of the painter's body formaking simultaneous and continuous strokes or lines in allthree-dimensions is detected by a system of sensors providing an inputinformation about free motion into the at least one input control devicefor digital painting or drawing within 3D virtual canvas, wherein thesystem of sensors are chosen from a group of sensors consisting of: aone, two or three-axis force sensor; a pressure sensor; a one, two orthree-axis tactile sensor; a one, two or three-axis linearaccelerometer; a one or two-axis tilt sensor; a one, two or three-axisgyro sensor; a one, two or three-axis angular accelerometer; a one, twoor three-axis magnetometer; 6D or 9D Inertial Measurement Unit (IMU); analtimeter; an optical sensor; a photo-sensor array; an electromagneticsensor; an ultrasound sensor; a microphone; and combinations, whichprovide a multi-axis sensor system for determining linear and angularmotion parameters of the at least one part of the painter's body, whichhas been moved between recording of two virtual positions of the atleast one input control device within 3D virtual canvas.
 3. A methodaccording to claim 1, wherein the means for three-dimensional imagepresentation comprising a processor provide: means for receiving inputsfrom at least one multi-axis input control device for three-dimensionalimage presentation for painting of lines or strokes along and aroundthree axes X, Y and Z within XYZ virtual space and wherein at least oneinput is used by the means for three-dimensional image presentation forpainting or drawing lines or strokes along axis from the painter's eyestoward the digital electronic canvas and beyond by changing a horizontaldisparity between images for the right and left eye and their scaling,making three-dimensional paintings or drawings equally continuous in XYplane and XZ and YZ virtual planes; and means for using a motioncaptured by the at least one input control device for making lines orstrokes, and the motion is provided simultaneously and continuously inall three dimensions by linear and angular movements of the at least onepart of the painter's body; and means for description in digital formatof images for the right eye and the left eye for every and all instantpositions within 3D virtual canvas; means for complete two-dimensionalimages of a painting for the right eye and the left eye at any stage ofpainting as accumulation of all lines and strokes made up to that stage;and means for complete three-dimensional images of a painting includingits linear and angular position within 3D virtual canvas.
 4. A methodaccording to claim 1, wherein the continuous 3D virtual canvas ispresented as a temporary 3D virtual grid created by a system of linesrelated to a chosen system of coordinates within said 3D virtual canvas,and wherein said temporary 3D virtual grid or at least one selectedportion of it can be deformed, scaled, changed, adjusted, rotated orshifted relative to the painter, wherein the temporary 3D virtual gridallows the painter navigate placement and scale the size of strokes or3D sketching within initially empty 3D virtual space and also placestrokes to sides, to back, to top or bottom of a paintedthree-dimensional object, when a front view or the entire sketch of thepainted three-dimensional object is already defined and wherein thetemporary 3D virtual grid can be turned off and on at any time in aprocess of painting.
 5. A method according to claim 4, wherein at leastone selected portion of the temporary 3D virtual grid is dedicated as atoolbox for arranging and organizing multiple parameters andcharacteristics of the painting tools, color palette, painting processand its different steps, visual effects, setups, commands, wherein aselectable 3D toolbox is chosen from a group: rolodex, 3D stack ofsliding flat windows, rotating in different directions geometricalshapes like cube, cylinder, prism, pyramid, sphere, ellipsoid, thesurface of which is used for positioning the icons activatingcorresponding actions, 3D tree with a trunk, branches, sub-branches andleaves corresponding to desirable organizational structure.
 6. A methodaccording to claim 1, wherein the method is used for painting: panoramic3D pictures, wherein 3D canvas is surrounding the painter and paintercan paint either by walking with 3D headset viewer within 3D canvas andpainting with input control devices (“walking mode”) or by sitting in achair in front of a steady display and virtually rotating the grid ofthe 3D canvas around him sector by sector while painting portions ofpanoramic picture (“turntable mode”) and localized 3D pictures, wherein3D canvas is localized within certain area and painter can paint eitherby walking with 3D headset viewer around 3D canvas and painting withinput control devices (“walking mode”) or by sitting in a chair in frontof a steady display and rotating the grid of the 3D canvas around somevertical axis located within the 3D canvas verifying this rotation bymeans for three-dimensional digital vision while painting side by sidelocalized 3D picture (“turntable mode”).
 7. A method according to claim1 further comprising: providing zooming in or out of a selected portionof 3D virtual canvas at any stage of a painting process providing higheraccuracy of detail painting and better perspective of how it looks likefrom different distance for a viewer; freezing a position of a brushwithin interested area of 3D virtual canvas, when the brush is moved;providing switching accuracy of the brush to a higher value by eitherswitching the brush or adjusting input control on the brush, whichprovides required accuracy; providing a firm and stable surface forsupporting a hand with the brush, which provides more accurate motion ofthe hand and the hand's fingers; providing turning on the brush's motionand continue to paint within interested area of three-dimensionalvirtual space.
 8. A method according to claim 1, wherein the painter isusing multiple input control devices coupled to different parts of hisbody by different kind of coupling for controlling functions of adigital painting process chosen from a group consisting of: action,turning on and off, navigation of a cursor (object, image) withinvirtual painting space, scrolling, scaling, zooming, shadowing,screening, selecting, moving, rotating, deleting, restoring, saving,opening, closing, searching, setting up, previewing, undoing, clearing,repeating, pasting, finding, replacing, inserting, formatting, grouping,ungrouping, color selection, color mixing, assigning line or strokewidth, assigning brush size, assigning swatch size, assigning spongesize, assigning eraser size, assigning a canvas virtual position,assigning depth of a focal plane around an instant virtual position ofthe stroke within 3D virtual canvas, creating special effects, and acombination thereof.
 9. A method according to claim 1, wherein themethod is used for editing, enhancing, filtering and modifyingthree-dimensional photo images for development of new 3D games, based oncollective collaborative painting, for personalization of existing gamecharacters by adjusting, editing, correcting or complete redrawing orrepainting the existing game characters to player's taste within games,said method additionally comprising: providing a three-dimensional photoimage; using this 3D image, as a basis of system of coordinates,defining the 3D virtual canvas, within which photo image supposed to beedited; painting, drawing, editing, enhancing, filtering or modifyingselected areas within 3D virtual canvas with an at least one inputcontrol device and verifying the position with the means forthree-dimensional digital vision; providing complete modified and editedtwo-dimensional images of the 3D photo image for the right and the lefteyes; using left and right images for presentation of editedthree-dimensional photo image.
 10. A method according to claim 1 furthercomprising multiple three-axis input control devices, multiple means forthree-dimensional digital vision, multiple means for three-dimensionalimage presentation comprising a processor, all connected into a networkof multiple users collectively and simultaneously participating inthree-dimensional painting, drawing, 3D photo images editing, gaming,studying, brainstorming, and researching, wherein working withthree-dimensional imaging is required.
 11. A system for digitalcontinuous and simultaneous three-dimensional painting,three-dimensional drawing, three-dimensional cursor (object, image)navigating, and digital recording of three-dimensional painted andthree-dimensional drawn images, said system comprising: a digitalelectronic canvas having a screen or display with a physical surface andconfigured for presenting two pictures: one for a right eye and theother for a left eye; means for three-dimensional digital vision,through which a painter sees 3D images; at least one multi-axis inputcontrol device comprising: a system of sensors, which provide an inputinformation about free 3D motion of at least one part of the painter'sbody into the at least one multi-axis input control device for digitalpainting, or drawing or navigating 3D images: means for at least onekind of a coupling between at least part of the at least one multi-axisinput control device and at least one part of the painter's body, saidcoupling chosen from a group consisting of: mechanical coupling, opticalcoupling, electromagnetic coupling, sound coupling, ultrasound coupling,and a combination of two or more thereof; means for three-dimensionalimage presentation comprising a processor; wherein inputs of the atleast one multi-axis input control device are used by the means forthree-dimensional image presentation for painting of lines or strokesalong and around three axes X, Y and Z within XYZ virtual space, meansfor creating a continuous 3D virtual canvas comprising the display'ssurface of the digital electronic canvas and a volume that includes thesurface by digitally changing a value and sign of horizontal disparitybetween two images for the right eye and the left eye and their scalingon the digital electronic canvas corresponding to instant virtualdistance between the painter's eyes and an instant image within thevirtual 3D canvas; wherein a resolution Δ of continuity of changing ofthe virtual distance Z between the painter and the virtual images within3D virtual canvas is defined by a size p of a pixel on the digitalelectronic canvas in horizontal direction and by a distance d betweenpupils of the painter's eyes according to an expression: Δ≈2p Z/d;wherein a motion captured by the at least one multi-axis input controldevice is used by the means for three-dimensional image presentation formaking lines or strokes, and the motion is provided simultaneously andcontinuously in all three dimensions by movements of the at least onepart of the painter's body; wherein a simultaneousness of appearance ofsaid similar strokes or lines on the images for the right and the lefteye is limited by a smallest time interval equal to an invertedfrequency of refreshment of frames on the digital electronic canvas andwherein a motion for making strokes or lines in all three dimensions isprovided simultaneously and continuously in all directions within 3Dvirtual canvas by free moving the at least one part of the painter'sbody; wherein the means for three-dimensional image presentationprovides a description in digital format of images for the right eye andthe left eye on the digital electronic canvas for every and all instantimages made within 3D virtual canvas; wherein the means forthree-dimensional image presentation provides complete two-dimensionalimages of a painting for the right eye and the left eye at any stage ofpainting as accumulation of all lines and strokes made up to that stage;and wherein the means for three-dimensional image presentation providescomplete three-dimensional images of a painting including linear andangular position within 3D virtual canvas as accumulation of all linesand strokes made to the stage of completion and the means forthree-dimensional image presentation provides two-dimensional left andright images to the means for three-dimensional digital vision forpresentation of three-dimensional painting.
 12. A system according toclaim 11, wherein said digital electronic canvas having a screen ordisplay and capable to present two pictures for a right eye and a lefteye is chosen from a group consisting of: projection screen for digitalprojection, 3D television (TV), digital autostereoscopic monitor,computer monitor, display of a mobile device, eye glasses or headsetwith two micro-displays for a right and a left eye for a directprojection of right and left images into a retina of an eye.
 13. Asystem according to claim 11, wherein said means for three-dimensionaldigital vision are chosen from a group consisting of: polarized glassesfor viewing only one out of two images by each eye, images are projectedon a screen or a monitor with corresponding polarization for each eye;color glasses for viewing two corresponding images; shutter glasses,time-splitting canvas on two canvases and synchronously with shutterglasses switching canvases between left and right eye; autostereoscopicsystems; eye glasses or headset with two micro-displays for a right anda left eye for a direct projection of tight and left images into aretina of a corresponding eye.
 14. A system according to claim 11,wherein sensors for the system of sensors are chosen from a group ofsensors consisting of: a one; two or three-axis force sensor; a pressuresensor; a one, two or three-axis tactile sensor; a one, two orthree-axis linear accelerometer; one or two-axis tilt sensor; a one, twoor three-axis gyro sensor; a one, two or three-axis angularaccelerometer; a one, two or three-axis magnetometer; a 6D or 9DInertial Measurement Unit (IMU); an altimeter; an optical sensor; aphoto-sensor array; an electromagnetic sensor; an ultrasound sensor; amicrophone; and combinations thereof, which provide a multi-axis sensorsystem for determining linear and angular motion parameters of the atleast one part of the painter's body, which has been moved betweenrecording of two positions of the at least one multi-axis input controldevice within 3D virtual painting space (canvas).
 15. A system accordingto claim 11, wherein the means for three-dimensional image presentationcomprising a processor are configured to: provide receiving inputs fromthe at least one multi-axis input control device for three-dimensionalimage presentation for painting of lines or strokes along and aroundthree axes X, Y and Z within XYZ virtual space and wherein at least oneinput is used by the means for three-dimensional image presentation forpainting or drawing lines or strokes along axis from the painter's eyestoward the digital electronic canvas and beyond by changing a horizontaldisparity between images for the right and left eye and by scaling theimages, making three-dimensional paintings or drawings equallycontinuous in XY plane and XZ and YZ virtual planes; and provide using amotion captured by the at least one multi-axis input control device formaking lines or strokes, and the motion is provided simultaneously andcontinuously in all three dimensions by linear and angular movements ofthe at least one part of the painter's body; and provide description indigital format of images for the right eye and the left eye for everyand all instant virtual positions of the strokes within 3D virtualcanvas; provide complete two-dimensional images of a painting for theright eye and the left eye at any stage of painting as accumulation ofall lines and strokes made up to that stage; and provide completethree-dimensional images of a painting including the painting's linearand angular position within 3D virtual canvas.
 16. A system according toclaim 13, wherein the eye glasses or the headset with two micro-displaysfor a right and a left eye for a direct projection of right and leftimages into a retina of an eye, additionally comprises sensors andactuators chosen from a group consisting of: multi-axis mechanicalmotion sensors, one or two photo-sensitive arrays for measuring movementof eye-balls of the painter, three-axis magnetometer for determiningrelative angular position of the painter and a painted three-dimensionalimage; altimeter; optical sensor; one or two photo-camera arrays formaking 3D photo-pictures or 3D video; electromagnetic sensor andtransmitter; ultrasound sensor and transmitter; microphone;speakerphone; and a combination of the sensors and actuators, whereinthese sensors and actuators provide an expanded functionality of theeye-glasses and headsets as either additional input control devices orpart of other input control devices for digital continuous andsimultaneous 3D painting, drawing and navigating.
 17. A system accordingto claim 16, wherein the eye glasses or the headset with twomicro-displays for a right and a left eye for a direct projection ofright and left images into a retina of an eye comprises a smart phone,which provides two micro-displays for a right and a left eye withincommon display of the smart phone and which also has two photo-camerason an opposite side to a display side of the smart phone, said twophoto-cameras are positioned along longer side of the smart phone at adistance from each other corresponding to an average distance betweenhuman eyes and wherein said eye glasses or said headset has opticallenses for viewing 3D photo-picture or 3D video including viewing in theprocess of shooting and recording the picture or video.
 18. A systemaccording to claim 17, wherein said system is configured to provide,after taking 3D stereo photo-picture or 3D video, editing, enhancing,filtering and modifying three-dimensional photo images, and to providetransmitting an edited 3D images via standard channel by the smart phoneused within the system, wherein a recipient can immediately watch 3Dimages using his/her cell phone that doesn't have two photo-cameras. 19.A system according to claim 16, wherein the eye glasses or the headsetwith two micro-displays for a right and a left eye for a directprojection of right and left images into a retina of an eye comprises auniversal tool, which provides two micro-displays for a right and a lefteye within common display of the universal tool and which also has twophoto-cameras on an opposite side to a display side of the universaltool, said two photo-cameras are positioned along longer side of theuniversal tool at a distance from each other corresponding to an averagedistance between human eyes and wherein said eye glasses or said headsethas optical lenses for 3D stereo photo-picture or 3D video and wherein anarrower size of the universal tool compared to a regular smart phoneallows position universal tool with the display and lenses for viewingthe pictures on a display such that they would not obstruct eyes fromviewing surrounding.
 20. A system according to claim 11 furthercomprising multiple multi-axis input control devices, computers anddigital electronic canvases connected into a network of multiple userscollectively and simultaneously participating in three-dimensionalpainting, drawing, 3D photo images editing, gaming, studying,brainstorming, or researching, wherein working with three-dimensionalimaging is required.